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ABSTRACT 


Sign language is a complete, complicated language that combines hand 
motions with facial expressions and body postures to create signs. Hand 
geometry, movement, or orientation of the arm, hands, or body, as well as 
facial emotions, can all be used to convey a speaker's views. A sign language 
usually provides a sign for whole words. In each country, the sign language is 
different. The usage of signs varies by nation, such as the sign language used 
in Tanzania versus the sign language used in India. The major goal of this 
project is to make it possible to translate signals into sound. The suggested 
system is a smart sign translator for those who are deaf or hard of hearing 
who communicate using sign language. This technology uses specially built 
sensor gloves that are connected to the system while a disabled person 
wearing the gloves produces signs that are converted into sound, making it 
understandable to others. The Proposed system is portable and easy to handle. 


Index Terms: Communication, Signs, Dumb and Language 


1. INTRODUCTION 


The most crucial aspect of life is communication. Deaf and mute persons 
communicate through sign language, which combines hand forms and 
motions instead of sound to convey meaning (Kurtz et al., 2017; Tongue et al., 
2005). Words and sentences are communicated to an audience via signs. In 
sign language, a gesture is a distinct movement of the hands that creates a 
specific shape. Whole words are frequently represented with a sign in sign 
language (Buvaneswari et al., 2020). However, it is discovered that the hand 
plays an active role in sign language, yet hand gesture identification is a 
difficult task due to its complicated articulated structure, which consists of 
numerous connected linkages and joints. There is no standardized sign 
language for all deaf people across the world (Nagpal et al., 2015). Yet, sign 
languages are not universal, as with spoken languages, these differ from 
region to region. The Proposed system is portable and easy to handle (Kadam 
et al.,). The suggested system is a sign language recognition system for people 
who are deaf or hard of hearing who communicate using sign language. This 
system employs specially built sensor gloves that are connected to the system 
and worn by a disabled person who makes signals and motions, which are 
then translated into sound/voice on the audio playback and written words on 
the display. 
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Problems in impaired-normal individual communication 

There are numerous methods for a stupid person to try to connect with normal people in society; but, if the individual does not 
recognize the dumb sign language, this can be a major disadvantage. A sizable portion of the population is unaware of the 
accessible sign language. 


Figure 1 Signs used in sign language communication 


Causes of the problem 

Muteness is most commonly caused by difficulties with the parts of the human body required for human speech (the esophagus, 
vocal cords, lungs, mouth, or tongue), as well as damage or injury to Broca's region, which is located in the left inferior frontal lobe 
of the brain. 


Objectives 

The primary goal of this project is to create an electronic device that can convert a deaf person's sign into voice/sound and display 
the words in textual form. The project will be made possible by a flexible sensor, whose angle of bend will determine which word 
from a microcontroller should be spoken utilizing audio playback. 


RELATED WORK 

For the recognition of various hand gestures, many researchers have employed various methodologies, which have been used in 
various sectors. Vision-based approaches, data glove-based approaches, soft computing approaches such as Artificial Neural 
Networks, Fuzzy logic, Genetic Algorithms, and others such as PCA, Canonical Analysis were among the approaches used (Nikam 
and Ambekar, 2016). Hand segmentation approaches, feature extraction approaches, and gesture recognition approaches are the 
three basic categories in which the entire methodology can be classified. Various sensors, an LCD, an appropriate microcontroller, a 
voice recorder, playback, and a power source are used in all of the above approaches (Singha and Das, 2013; Pigou et al., 2014). The 
static hand gesture gadget is shown. Any tiny change in finger orientation will be interpreted as a signal, and the microcontroller 
will evaluate it and use its memory to deliver the needed output in the form of speech (Bhuvaneshwari and Manjunathan, 2020; 
Islam et al., 2017). The proposed method will allow deaf persons to communicate with regular people via sign language, with hand 
movements being transformed into relevant speech messages. The main goal is to create an algorithm that uses a digital sign 
processor to convert dynamic motions produced by the user into voice in real time. 
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2. MATERIALS AND METHODOLOGY 


Field survey of the Problem and its Impact 
According to the results of the field survey, we discovered a number of issues that affect the dumb impaired in society, which we 
want to address. The following are some of the difficulties that an impaired person may face. 

Inability to communicate between disabled and non-disabled people. If you don't know what the signs mean, it's tough to 
understand them. In addition, most of them have limited access to schools because they believe that education is of little value to 
them. Again, because they are unable to communicate, the majority of them are unemployed. They have fewer opportunities to find 
work. No one wants to hire someone who is illiterate. Moreover, they are derided in society because of their deafening silence 
(failure to speak). In addition, when compared to regular people, they receive less attention. 

The following are their consequences because of the aforementioned social issues. 

a. Because dumb people have limited or no access to schools, they have a high level of illiteracy. 

b. Discrimination in society should be increased. 

c. They are regarded as ineffective members of society. 

d. Because they are unable to obtain legal employment, the majority of them end up as beggars or criminals. 

e. In society, they tend to have fewer friends or none at all. 

f. They require assistance in interpreting whenever they need to express themselves, such as in hospitals where a translator is 
required for a discourse between a doctor and a deaf patient. 


Existing solutions from field survey 

The dumb impaired person uses a variety of methods to try to communicate their message to society in the same way that other 
living organisms do. The solution can be self-taught or one that is agreed upon country-wide and obtained through learning or 
taking lessons. The following are some of the available options: 

Sign language is a language in which meaning is communicated through manual communication. This can include using hand 
gestures, movement, finger, arm, or body position, and facial emotions all at the same time to convey a speaker's views. Sign 
languages and their spoken counterparts can share substantial similarities (such as ASL and American English). However, to 
encourage speed and flow in speaking, grammar and sentence structure may differ. Both spoken and signed communication are 
considered types of natural language by linguists, implying that they originated from an abstract, long-term aging process and 
evolved without rigorous preparation. All countries, including Tanzania, use their indigenous sign language. As a result, if you 
were from Tanzania and came to Kenya, your Kiswahili Sign Language would be incompatible with Kenyan Sign Language, 
despite the fact that both nations’ hearing communities could speak and understand Kiswahili. Because of the many settings they 
come from, there is the usage of gestures in this scenario, as well as the use of signs. Although the gestures are not official, some 
deaf people use them as a means of making themselves recognized in society. Most of those who use gestures are untrained in sign 
language and rely on them to communicate with others and make acquaintances. Total communication entails the simultaneous use 


of several body organs, such as fingers, facial gestures, and body shape. 


A proposed solution 

The proposed system is a portable, easy-to-use, and comfortable translation device for those who are deaf or hard of hearing and 
communicate using sign language. This system involves the use of specially designed sensors fitted on gloves with other electronic 
components and then worn on the hand by the dumb and will enable the translation of any sign formulated by the dumb impaired 
individual into voice and text. Any finger bend will cause the system to output a programmed word in the microprocessor. For this, 
will be voiced on audio playback and displayed as text on the LCD (display). 


Merits of a proposed solution 
The general advantages of the “SMART SIGN TRANSLATOR FOR DUMB” project are that; 

To provide a simple means of communication between native sign language speakers and the rest of the population. To make 
them feel less disregarded in society by allowing them to talk with the use of a sign translator. It may save lives in some situations 
because not everyone can understand the sign used by the deaf in distress, but everyone can comprehend voice and text. Because 
our method is portable and simple to use, a person who is unfamiliar with sign language does not need to be taught how to 
translate the signs; instead, all he or she needs to do is listen to the voice and comprehend the message conveyed by the dumb. 
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The feasibility of the proposed solution 

The proposed method is more practicable by a bigger proportion since it uses the entire word for a tilt of the flex sensor, minimizing 
the time delay of sending the message. Rather than employing sign language alphabets, the system not only reduces speech 
delivery time but also saves the dumb energy. The system is cost-effective due to the less cost components that make up the 
suggested solution's foundation. 


A review of system operation 

As shown below, the system is designed using a block diagram that includes all of the critical components of the smart sign 
translator system, including the flex sensor, accelerometer, microcontroller, regulated power supply, LCD (Liquid Crystal Display), 
audio recorder, and playback see figure 2 below. 


POWER SUPPLY (RPS) 


anetce MICROCONTROLLER DISPLAY 
SENSOR -——— 


VOICE RECORDER AND PLAYBACK 


Figure 2 diagram of the system 


A 4.2-inch flex sensor is used in this system to receive data from the glove, which is controlled by a microprocessor. This sensor 
generates a signal that corresponds to finger flexures and hand motion. The analog value from the sensor is received via a 10-bit 
ADC incorporated into the PIC microcontroller. An ADC converts analog to a digital value and stores the value in the buffer. After 
that, the controller compares the static data with the digital value to identify the gesture. According to the finger, movements the 
microcontroller plays the voice (speech) see the flowchart in figure 3. The APR9600 is a single chip that stores high-quality voice 
recordings and non-volatile flash memory with a 40- to 60-second replay capability. APR can send many messages in a random or 
sequential order, and designers can change the storage time based on user requirements. A microphone amplifier, output amplifier, 
and AGC circuit are all built within the chip. The Auto Rewind and standard options are available in TAPE mode. The APR's six 
pins are utilized for voice storage and playback, with each pin playing the voice for 60 seconds. 
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HAND GESTURE 
SENSOR DATA 


SIGN RECOGNIZED 


VOICE OUTPUT 


Figure 3 System flow chart 


3. RESULTS AND DISCUSSION 


Comparison of Analytical and Simulation Results 
As expected, the texts of each coded word were shown in real time. Similarly, some of the words were displayed with only a slight 
change in flex resistance. Minimal storage capacity of the microcontroller lead only to the display of the first three words, which are 


“it’s time to take my pills”, “can you get me today’s newspaper”, “get me some water to drink”. 
For further understanding, see figure 4 and 5 below. 


Flex sensors 


ay > Ll = 
AH =i PIC16F877A | TTT | 
[r wee ane = | 
eH Hill 


Voice recorder 
and Audio playback 


Figure 4 Simulation diagram of smart sign translator 
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Figure 5 Device picture of smart sign translator 


4. DISCUSSION 


Humans are born with the ability to see, hear, and interact with their surroundings. Unfortunately, some persons are differently 
abled and are unable to fully utilize their senses. These persons use other forms of communication, such as sign language. When 
persons in the deaf and dumb populations try to interact with others, especially in their educational, social, and professional 
situations, this presents a tremendous hurdle. To cross this communication gap, a sophisticated gesture recognition or sign 
language detecting technology is required. The technique helps unintelligent individuals overcome their time constraints and 
enhance their demeanor. When compared to the existing system, the proposed arrangement is more compact and portable. This 
technique turns the language into a passing voice that is easily understood by both blind and elderly individuals. To assist deaf 
persons, the language is translated into some form of text that is exhibited on a digital display screen. This approach, is useful in the 
real world for the deaf and dumb who are unable to converse with others. The gesture recognizer in this project is unique in that it 
can function as a stand-alone system that has been used in everyday life. It is also effective for speech impaired and paralyzed 
individuals (those who are unable to talk properly), as well as Intelligent Home Applications and industrial applications. 


5. CONCLUSION 


A sign language (also signed language) is a language that conveys meaning primarily through manual communication rather than 
acoustically communicated sound patterns. This can include combining hand forms, orientation and movement of the hands, arms, 
or body, and facial expressions to portray a speaker's thoughts in a fluid manner. Sign languages and spoken languages 
(sometimes known as "oral languages" because they are based mostly on sound) have many similarities, which is why linguists 
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consider them to be natural languages. However, there are also notable distinctions between signed and spoken languages. They 
should not be confused with nonverbal communication, such as body language. Humans are born with the ability to see, hear, and 
interact with their surroundings. Unfortunately, some people are physically unable to use their senses to their full potential. These 
persons use other forms of communication, such as sign language. When persons in the deaf and dumb communities strive to 
obtain their fundamental requirements with the support of others, especially in their educational, social, and professional situations, 
this presents a big hurdle. As a result, an advanced "SMART SIGN TRANSLATOR" is required to assist people in obtaining such 
demands at any location in society. The system's flexible sensors were used to design the sign translator. 


Future work 

The aim to build a system that will assist in the translation of sign language into a voice for the deaf drove this project. In the 
current situation, communication is limited to sign language, which some people find difficult to comprehend. Given that people 
who are deaf or blind are often overlooked in society since they can communicate and take a great deal of care to understand, the 
method we imagine is begging to be used. This initiative, of course, has barely scratched the surface. Our project is very much a 
proof-of-concept, as the design simplifications and implementation limits reveal. In order to create the desired output, a practical 
autonomous smart sign translator necessitates a collection of electronic equipment such as sensors, microcontrollers, audio 
recorder, and playback and display working together. Furthermore, such a system necessitates the use of a glove that will serve as 
the device's wearing structure. Future work should include as many words as feasible by attaching an extra memory module to the 
device, which will provide more storage. As a result, the number of words will be increased, the sign translator unit will be more 
customizable, and the region of coverage will be expanded. 
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